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STATISTICAL INFERENCE 


Retrieval of information by indirect observations 

x = unknown interesting quantity 
y = observable 


Problem: Estimate variable x based on observed y. 
Example 1: (Interdependence derived from physics): 
x = temperature of the air 
y = length of a metal rod + noise 


Model: 


y = Kx + e. 


Example 2: (Interdependence derived from previous ob¬ 
servations): 


x = temperature of the air 
y = time of the day 


Scatter plot of previous observations. 






Fundamental consepts: Joint probability distribution, con¬ 
ditional distribution 



Figure 3: Contour plot of a Gaussian distribution. Probability of the 
random variable to reside in the shaded rectangle is the integral over 
the set. 



Figure 4: Gaussian distribution. The random variable X 2 is fixed at 
the value X 2 =2.5. The conditional distribution of the variable X\ is 
obtained by cutting the joint probability distribution alon X 2 = 2.5. 
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STATISTICAL INVERSION THEORY 


1. Physical quantities are modelled as random variables. 

2. Random variables define probability distributions. 

3. The solution of an inverse problem is a conditional 
probability distribution (conditioned on the obser¬ 
vation) called posterior distribution. 

4. Single estimate calculation is based on the posterior 
distribution. 

The most popular estimates are maximum a posteriori 
estimatorand conditional expectation. 



Figure 5: The maximum and the expectation are marked. Notice that 
due to the skewness of the distribution, they do not coincide. 
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The statistical model should contain 

1. The physical model interrelating the quantities (if 
available). 

2. Information of the statistics of, noise, modelling 
errors etc. 

3. Prior information of the quantities (based on pre¬ 
vious experience, physics, common sense etc.) 
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Example: Electrical Impedance Tomography. 

Attach contact electrodes on the surface of a body. 

Inject electric currents, measure voltages. 

Estimate the resistivity (or impedance) distribution in a 
body from these estimates. 

Unknown : Resistivity p = (pi, P2, • • •, Pn )• (Pixels) 

Data : Voltages V = (Fp Vp ... , Vl), currents I = (/p A, ■ ■ ■ 
Model (Ohm’s law): 

V = i?(p)/ + n, 

where i?(p) is a resistance matrix computed from a diffe¬ 
rential equation (e.g. Finite Element Method) 


n = measurement noise. 



Statistics'. Noise is a random variable with probability den¬ 
sity 7r noise (n). 

Assume for a moment that p were known. Then, V is a 
random variable with probability distribution 

7T(V | p) = 7T noise (V - R(p)I). 
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Prior distribution of p is i r pr (p). 

Contains all the information of rho known to us prior to the 
measurement (e.g. positivity, boundedness, possible smooth¬ 
ness, anatomical information). 

If p is independent of n, probability theory says that the 

joint probability distribution of p and V is 

7 t(p, V) = 7T pr (p)n noise (V - R(p)I). 
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Posterior distribution = Bayes formula, 

n(p | V = ^measured) ^ ^(P’ ^ = ^measured) 

= Mp) ^hioise (^measured R(p)I) 

= ^post (p) • 

In statistical inversion theory, this is the complete solution 
of the inverse problem. 
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ESTIMATORS 

Maximum a posteriori estimator (MAP): Find p = pmap 
such that 


^post(pMAp) — max 7Tpost(p)- 


Conditional expectation: Calculate 

T>= / P n pos t(p)d N p, 

J rN 

i.e., componentwise (pixelwise), 

Pj = [ Pj TTpost {p)d N p, l<j<N. 

J rn 
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PROBLEMS 

The focus of the current research is on two questions: 

1. How to implement prior information? 

2. How to compute single estimators? 
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PRIORS: GENERAL SCHEME 
Write 


V (p) ~ e AM , 

where A(p) is a function that is small for typical admissible 
resistivities, and large for typical unadmissible resistivities. 

Example: MRI-guided EIT: Admissible conductivities jump 
across organ boundaries, unadmissible conductivities are 
smooth across the boundaries. 
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ESTIMATORS: GENERAL SCHEME 

Given 7r pos t(p), generate a large ensemble of samples 


that are distributed according to the probability density 
TTpostO)- Approximate 


P= p ir post {p)d N p 

Jrn 

- i±& 


Ensemble generation by Markov Chain Monte Carlo (MCMC) 
methods. 
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Figure 7: Gaussian distribution with positivity constraints, 100 
sample points by Gibbs sampler. 


EXAMPLE (continued): 
Gaussian additive noise: 

TTnoiseW ~ exp 



where C G M. LxL is the noise covariance matrix. 


n(V | p) ~ exp - R(p)I)' x C hr - R(p)I^j . 
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Selection of the prior: Try 


f N 

7Tpr(p) ~ exp I -a I Pn - p 0 \ 

\ nM: 

It turns out that the function 

N 

A (P) = ^2\pn~ Po\ 

I) 1 

is small, when only few pixels differ from po- 
Suitable e.g. for small tumor detection. 

Discussion of results: See 

J. Kaipio, V. Kolehmainen, E. Somersalo and M. Vauhko- 
nen: Statistical Inversion Methods in Electrical Impedance 
Tomography. (To appear in Inverse Problems , 2000) 
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